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ABSTRACT 


With the increased use of analytical data from multiple sources, the contributing laboratories are 
encouraged to participate in as many interlaboratory studies as possible to ensure data quality 
and comparability. It becomes necessary to develop techniques that will minimize the analytical 
workload and yet provide a comprehensive, rapid evaluation. Also, the evaluation protocol 


should present the study findings in a form that is easily understood by all participants. 


The graphical procedure described in this report requires only two samples to be analyzed by 
each participant. It is automated. The graphical presentations enable each participant to 
‘visualize’ their performance. The method is also able to distinguish between different types of 
errors and identify possible cause(s) of error(s) seen in laboratories not performing 


satisfactorily. 
This report also identifies possible remedial action for the different types of errors. 
The historical background and the theory are discussed in section 2. The various trigonometric 


calculations are described in section 3. Operational instructions, listing and explanations of the 


computer programs are detailed in section 4. 
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1. INTRODUCTION 


Interlaboratory studies play an increasingly important role in data quality assessment. 

Many projects today by their sheer size and extent use data from multiple sources. In 
such cases, data comparability becomes an important issue to scientists, engineers and 
other analytical data users. Interlaboratory studies are often used not 
only to evaluate individual laboratory performance, but also to assess the quality of the 
database as a whole. 


Any valuable interlaboratory study must include as many laboratories as possible. A 
study that involves minimum laboratory analytical work will encourage more participants. 
Other attractive features of any study are simplicity of the study protocol, timely release 
of study findings, and presentation of results in an easily understandable format. 


Study evaluations should identify the presence of determinate error(s) observed in a 
laboratory’s data and suggest possible causes and solutions. This will prompt the 
laboratory to take immediate remedial action. In other words, an effective interlaboratory 
study must lead to an appropriate action whenever warranted. 


The above issues were the main consideration in developing the procedure that is 
described in this document. 


The classical Youden’s two-sample graphical procedure for evaluation of interlaboratory 
data’ was the starting point of this development. King’s extension? provided the means 
of diagnosis and assignment of possible causes of errors. The King - Selliah procedure 
(K-S procedure) is essentially an effective combination of the above two approaches, 
automated to meet the goals specified earlier. 


The K-S procedure allows the laboratories to be evaluated with respect to either targets 
(designed values) or consensus values. The graphs enable each participant to visualize 
their performance. The tabular ‘output’ suggests the type of error whenever a laboratory 
is flagged as not performing satisfactorily. The evaluation is fully automated allowing 
timely release of study findings. 
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2. THEORY AND DISCUSSION 
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Many procedures have been developed to evaluate interlaboratory data. Among these, 
Youden’s graphical procedure’ is by far the simplest. It requires each participant to 
perform a single analysis on each of two samples of similar concentrations. The 
graphical evaluation ‘output’ enables each laboratory to visualize their performance in 
relation to the rest of the participants and take remedial action if warranted. Extensive 
statistical knowledge and computation is not required to estimate bias and imprecision. 


In the Youden graphical procedure the FIGURE 2.1 
result of one sample is plotted against the YOUDEN PLOT 
result of the other sample. This 
presentation distinguishes between 
random and systematic errors affecting 
the results. The graph is divided into four 
quadrants by two lines representing the 
median value ( or the target if known) of 
each sample (Figure 2.1). In the absence 
of any systematic errors the results will 
spread randomly between the four 
quadrants. However in actual practice the 
majority of the results are located in the 
lower left and upper right quadrants. In 
fact they spread out in a characteristic 
pattern along the 45 degree line (which 
only passes through the origin if the 


samples are exactly equal). 10 25 40 
Sample 1 


Sample 2 





The distance between the point of 

intersection of the two median (or target) lines and the mark representing the laboratory 
result is a measure of the total error of the results. The distance from the cenire along 
the 45 degree line gives the magnitude of the systematic error and distance perpendicular 
to the 45° line is a measure of the random component of the error (repeatability). 


It is customary to draw a circle with its centre coinciding with the point of intersection of 
the two median (or target) lines and a radius of some factor of appropriate standard 
deviation. The standard deviation itself is derived from the perpendicular distances. 
Depending on the factor chosen, the circle will represent acceptable repeatability. 
Laboratories that show a definite bias will tend to distribute along the 45 degree line (see 
Figure 2.1) outside this circle. 
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While this approach is useful in identifying laboratories that have problems and in 
distinguishing the random and systematic component of the total error, it does not provide 
information as to the type of the systematic error. Biases in analytical measurements 
may be induced by errors in setting the calibration intercept or slope, or by under- 
recovery due to method inadequacies. Therefore any systematic errors revealed in the 
Youden pattern, may or may not be concentration dependant. 


An extended version of the Youden plots has been used in the Ontario Ministry of the 
Environment Laboratories (MOE) for the last 20 years to evaluate between-run control 
data’. Unlike Youden’s approach, in this procedure the sample concentrations are 
dissimilar. This leads to identification of two bias lines. One passes through the origin 
and indicates bias that is concentration dependant. The other is at 45 degrees if the 
diagram is scaled properly and indicates bias that is concentration independent. 


A similar approach has been used to evaluate a number of recent MOE initiated 
interlaboratory studies***. Using of two samples of different concentration provided the 
ability to differentiate between the two kinds systematic errors mentioned earlier. 


The K-S procedure being described here for interlaboratory data evaluation, while 
maintaining the simplicity of the Youden’s method provides much more useful information. 
It is based on an automated spreadsheet ‘macro’ and includes screening of data for 
extreme results (outliers), estimating the variability among laboratories for each sample, 
plotting data points on a X-Y chart, estimating average repeatability, setting up 
acceptance criteria, and interpreting results. The steps involved in this procedure are 
outlined at the end of this section (page 10). Each of the major components of this 
method is described in more detail below. 


Data Screening And Estimation of Between-Laboratory Vanability 


Since most interlaboratory data will include erratic and biased data points, it is necessary 
to screen the data before evaluating dependence of the precision on concentration. 
Inclusion of these data in the estimation of standard deviations will exaggerate the 
variability and will mask useful conclusions that otherwise may be drawn. 


The screening process starts with the assumption that the confidence interval for the 
higher concentration sample will often be about 10%. The standard deviation S,, is 
calculated using only data which falls within this range of the sample median. The 
calculation of S,, is reiterated several times to re-include all points which fall within 3x the 
current estimate of S,. (If the precision of the data were significantly tighter a smaller 
initial limit eg. 5% would be used). 
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The relative standard deviation of the high sample CV, (S,/Mean,,,,) is then used to 
obtain the initial estimate of the standard deviation S, of the lower concentration sample 
(CV, *Mean,,,). The data is again evaluated iteratively as in the case of higher 
concentration sample to obtain the final estimate of S, . 


The above two iterative processes represent an initial attempt to estimate the general 
dependence of standard deviation on concentration. 


When two samples of significantly different concentrations are used, two scenarios can 
occur. In the first case the variability of the higher concentration sample (S,,) is not 
significantly different from that of the other (S,) indicated by the ratio of individual 
standard deviations (S,,/S,) being less than two. This implies that the random error of 
most laboratories is limited by blank or sample preparation variability. 


The other situation occurs when the variability of the higher concentration sample (S,,) 
is significantly higher than that of the lower concentration sample (S,) as shown by the 
ratio of S,/S, being greater than two. This indicates that random variability may be 
concentration dependant. 


The evaluation procedures for these two cases differ slightly. When the variability is not 
dependant on concentration, the average repeatability (S,) is expressed in concentration 
units and the absolute results (in concentration units) are plotted in the K-S plots. When 
the variability is dependant on concentration the average repeatability (S,) is expressed 
in percent (i.e. actual result x 100 /median (or target)) and results are plotted on a 
relative scale (percentage recovery) in the graph. 


Plotting Data on a X-Y Chart 


In either case the result of one sample of each laboratory is plotted against the result of 
the other sample on a X-Y chart. This leads to a Youden type scatter plot. In a typical 
graph, in addition to the two median (or target) lines, there are two other lines that bisect 
the graph representing the slope (concentration) dependant error and the intercept (blank) 
dependant error. The slope error line passes through the origin and the point of 
intersection of the two median (or target) lines. The exact position of the intercept error 
line will depend on the ratio of the median (or target) concentrations of the two samples. 
Positioning the intercept error line will be discussed in section 3 (trigonometric 
computation). 
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Average Repeatability (S,) Estimation _ 


The process of estimating average repeatability (S,,) is outlined in the step-wise summary 
at the end of this section (page 10, 5.iii to 5.vi). Estimation of S, involves trigonometric 
computation of perpendicular distances (PD) from every data point in the graph to both 
bias lines (section 3), calculation of an initial estimate of repeatability Sy i,q ( of all PDs to 
the 45 degree line) and selection of appropriate PDs for the final estimation of 
repeatability (S,). 


If a laboratory is considered biased based on S,,,,, then the lesser of the two PDs is 
selected. Other-wise PD to the 45 degree line is selected. The selected PDs are 
screened for extreme values (values greater than 3 x S,.,4 are eliminated from the 
selection). The final estimate of S, is obtained by multiplying the average of all those 
PDs finally selected, by the Youden factor of 1.2533’. 


Acceptance Criteria 


The warning limits and control limits for repeatability are set at 2S,, and 3S, respectively, 
representing approximately 95% and 99% confidence intervals. 


Additional tolerance is required for the effect of variability in preparing and using 
standards on a day to day or among laboratory basis. The factor 1.5 for the ratio of 
acceptable reproducibility(S,) versus repeatability(S,) was chosen to provide a goal for 
interlaboratory reproducibility. A ratio (S/S,)* exceeding 2.3 (i.e 1.5° ) would be 
considered significant with a risk of error of less than 10%, 5%, and 1% respectively for 
10, 20, and 35 degrees of freedom. Results that exceed waming and control limits 
determined from this desired maximum interlaboratory reproducibility(S,) are deemed to 
be possibly or probably biased respectively. Since the reproducibility(S,) is 1.5 times the 
repeatability S, the warning and control limits for bias are set at 3S, and 4.55, 
respectively. 


Interpretation of Results 


The assessment of a laboratory in this type of evaluation is based on the location of its 
result on the graph. The Figure 2.2 identifies the various regions in a typical graph 
associated with the different types of problems that might be experienced by the 
participants in a typical study involving two samples of different concentrations. 


Laboratories with controlled repeatability but showing various degrees of bias will appear 
in the lower left and upper right quadrants. The two circles drawn in these diagram 
represent the warning limits for repeatability(S,) and reproducibility(S,). Those points 
within the outer circle but in the upper left and lower right quadrants (unshaded in Figure 
2) are unbiased but somewhat less precise. Those points within the circle but in the 
upper right and lower left quadrants are precise and acceptably biased. 
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FIGURENZE 


LOW SAMPLE 








Thus the area of acceptable performance in this diagram has taken the shape of a 
keyhole. All laboratories that lie outside this area have exceeded the respective warning 
limits. The term ‘acceptable’ throughout this discussion is based on the observed 
performance of those participants reporting comparable data. 


Laboratories that lie close to any of the two median (or target) lines (within 2S,) have one 
result correct and the other ‘in error. They are out of control. Laboratories that lie 
along the intercept error line show intercept dependant bias. Laboratories that lie close 
to the slope error line show concentration dependant bias. The proximity of a point to any 
of the above lines will determine the probability of an error corresponding to that type of 
line. For example if a laboratory lies within 1S, of the intercept error line it is highly 
probable that this laboratory has blank or intercept related bias. If the laboratory is 
between 1-2S,, from the intercept line, a blank problem is suspected. 


When a laboratory lies between an error line and the adjacent median (or target) line and 
not close to any of the above, it can be concluded that this laboratory is highly imprecise 


and biased (biased/erratic). 
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When a point is between the two error lines but not close to any of these lines it is not 
possible to identify the type of bias except to document as ‘high’ or ‘low’ bias. 
Laboratories that lie in the upper left or lower right quadrant and are outside the control 
limits for repeatability (3S,,) are very imprecise. The performance of such laboratories is 
considered erratic 


Summary 


The K-S procedure while maintaining the simplicity of Youden’s method provides much 
more useful information. It is able to differentiate between two main types of systematic 
errors. 


It is fast and amenable to automation. 

The screening of data to exclude erroneous data prevents overestimating the average 
repeatability achievable in a particular analysis and/or matrices. This information will be 
very useful for project quality planners and management. 

In a recent interlaboratory study* the majority of the laboratories that were flagged 
exhibited slope (concentration) dependant error. In such acase use of certified reference 
standards to validate the ‘in-house’ standards or use of the same source of ‘quality’ 
external standard will probably improve the data comparability. 

In another study” (phenolics) many participants displayed blank or baseline related bias. 
A better control mechanism for determining and correcting or eliminating such effects 
would be beneficial. 

BIBLIOGRAPHY 


À. Graphical Diagnosis of Interlaboratory Test Results, W.J.Youden, Industrial Quality 
Control, Vol XV, No 11, May 1959 


À Systematic Error: Identification and Control, Donald E. King, in preparation. 


3: Interiaboratory Study 89-6: Metals, Nutrients & Phenolics (4-AAP) in Reagent 
Water, Sathi S.Selliah, Queen’s Printer for Ontario, 1991. 


4. Interlaboratory Study 90-5: Volatile Organics (MISA Test Group 17 &18) in 
Reagent Water, Sylvia Cussion, Queen’s Printer for Ontario, 1992. 


5; Interlaboratory Study 90-1: Total Cyanide in Reagent Water & STP effluent, Sathi 
S.Selliah, Queen's Printer for Ontario, 1992. 


Page 9 


Sequential Outline of The K-S Procedure 


1. Distribute two samples of different concentrations among a number of laboratories for 
analysis/measurement using their current methodology. 


2. Enter data on LOTUS 123° spreadsheet. 
Calculate median (L,,,H,,), means and standard deviations for each sample. 
Tabulate data and return to participating laboratory analyst for verification. 
Correct database if transcriptional errors were reported. 


3. Evaluate high sample data: 


i) reject all results which differ from the median (H,,) by more than 10% 
ii) calculate median (H), mean and standard deviation (S,) 
iii) re-include data if within 3 times S, 
iv) reiterate ii) and iii) until no further data is included 
v) calculate relative standard deviation of the final selected data 
(CV,) 


4. Evaluate low sample data: 


i) use 3 x CV, x (L,,) to exclude possible outliers 

ii) calculate median (L), mean and standard deviation (S) 
iii) reinclude data if within 3 times S) 

iv) reiterate ii) and iii) until no further data is included. 


5. Determine paired sample performance criteria: 


i) examine the ratio of S,/S, and if: 
<2 use data as reported in concentration units 
otherwise convert to % recovery based on expected value if known (otherwise use median 
values (H,L)) 
ii) prepare paired sample scatter diagrams of all data 
iii) calculate perpendicular distance from each point to the two error lines (PD.46, PDinercese)- 
iv) determine the median (PD, sn) Of all perpendicular distances to the appropriate 45 degree 
line (intercept error line or slope error line for absolute or relative scale respectively) 
v) calculate the bias for each laboratory (ie the distance from the centre along the appropriate 45 degree 
line) and if: 
<4.5 times PD, gan Select the PD to the 45 degree line 
other-wise select the lesser of the two PDs (PD... OF PDimercoc) 
vi) determine the average of all selected PD values less than 2.5 times the PD,,.a, and use this average 
to estimate the average repeatability S, (see reference 1). 
vi) set warning limits for repeatability = 2 times S,, 
viii) set control limits for repeatability = 3 times S,, 
ix) set warning limits for possible bias = 3 times S,, 
x) set control limits for possible bias = 4.5 times S, 


6. Code performance based on location of points on the diagram using LOTUS 123° program: 
i) in upper left or lower right quadrant (erratic) 
ii) in lower left or upper right quadrant (biased low or high) 
iii) on horizontal or vertical axis (out of control) 
iv) on diagonal line through origin (slope or standard problems) 
v) on diagonal line not through origin (intercept or blank problems) 


7. Prepare summary table of performance assessment. 
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3. TRIGONOMETRIC COMPUTATIONS 
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NOTE: In this section medians are assumed to be the basis of evaluation. If targets were 
choosen to be the basis of evaluation then the word ‘target’ should be substituted for the 
word ‘median’ through-out this section. 


For this discussion the higher concentration sample will be plotted on the X-axis and the 
lower concentration sample on the Y-axis. The MOE (LOTUS 123%) program automatically 
assigns the following scales: 


Absolute scale X-axis 0 - 2*median,,,, 
Y-axis 0 - 2*median,,,, 
Relative scale both axes 0% - 200% 
FIGURE 3.1 


LOV CONCENTRATION SAMPLE (PP) 





In the diagrams (Figure 3.1 & 3.2) the point R corresponds to a pair of results and B is 
point of intersection of the two median lines. The intercept error line meets the X-axis at 
position T. The slope error line passes through the origin and B. Point D corresponds 
to higher concentration sample median. 


POSITIONING THE INTERCEPT ERROR LINE 





In all cases the intercept error line originates at T and passes through B. 


Positioning of T is relatively easy for graphs in absolute scale. T is positioned such that 
TD equals BD (in concentration units). 
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The position of T in relative scale is given by 


100 (HAE ge PET E ) 
Median, 


TRIGONOMETRIC COMPUTATION OF THE MAGNITUDE OF RANDOM ERROR 


The computation of the measure of the random error for each laboratory will depend on 
the position of the laboratory on the graph and on whether absolute or relative scales 
were used. These computations are best understood by referring to Figures 3.1 & 3.2. 
Figure 3.1 refers to those cases where the majority of the participants show errors that 
are slope independent. Figure 3.2 illustrates the case where errors are slope dependant. 
The length of RB is the magnitude of the total error. The perpendicular from R to the 
lower concentration sample median line falls on point C. | and S are points on the 
intercept error line and slope error line respectively where the appropriate perpendicular 
lines (Rl, RS) from R meet these two lines. RI and RS corresponds to PDintercepe and 
PD sone respectively (section 2). Lower case and upper case characters in these diagrams 
correspond to pairs of results located in different zones on the graph. 


The following general equations apply to all cases. 


RC=ABS(MEDIAN, ow saupre - RESULT, ow.sauri à 
BC=ABS(MEDIANyicy-saupus = RESULT you-sampcd 
RB = V (RC? + BC?) 


LRBC = 8 = arc sin (2E 
RB 
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FOR ERRORS NOT CONCENTRATION DEPENDANT (FIGURE 3.1) 


Let ZRBI = p 


Let LSBC = à 
median 
d = arc tan (——__" =") 
MEGAN gp sample 


Let /SBR = 4 


When a laboratory appears in the lower left or upper right quadrant, 


sin p = sin (ABS (45 - 4)) 


sin n = sin (ABS ($ - 6)) 


When a laboratory appears in the upper right or lower left quadrant, then 


sin p 


sin (6 + 45) 


sin n 


sin (b + 8) 


In both cases, Intercept random error(Ri) and Slope random error(RS) are given by: 


Al = RB « sin p 
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RS = AB « sin n 


The bias BI (along the 45 degree line) is given by 
BI = RB + cos p 


FOR ERRORS CONCENTRATION DEPENDANT (FIGURE 3.2 





FIGURE ‘3. 2 


LOW CONCENTRATION SAMPLE (® OF TARGET) 





HIGH CONCENTRATION SAMPLE (%X OF TARGET) 


let LABS = À 


Let LIBD =a 
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DT 
= tan(—— 
o = arc (FB 


Let ZIBR = w 


When a laboratory appears in the lower left or upper right quadrant, 


sin À = sin (ABS{45 - 8)) 


sin w = sin (ABS((90-(c+6))) 


When a laboratory appears in the upper right or lower left quadrant, 
sin À = sin (45 + @) 


sin w = sin (80-ABS(o - 8) 


In both cases, Intercept random error (RI) and Slope random error (RS) are given by: 


Ril = RB « SIN w 


RS = RB + SIN 


The bias BS (along the 45 degree line) is given by 
BS = AB * cos i 
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4. LOTUS 123° PROGRAMS 
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4.1 Overview 


LOTUS 123® (version 2.1 or higher) is used to automate the majority of the steps involved 
in the evaluation procedure. A basic operational knowledge of LOTUS 123° is required 
to run the series of programs associated with this evaluation. In addition, a knowledge 
of LOTUS FREELANCE® or similar graphics software capable of importing ‘pic’ files is 
essential to generate graphs with ‘acceptable performance’ zones demarcated. . 


The step-wise summary (section 2) describes the various steps involved in the evaluation. 
These steps are grouped for convenience into a number of small programs. The inter- 
relationship among these programs is presented in the form of a flow diagram (Figure 
4.1). Each of these programs is described below. 


FIGURE 4.1 


CODE-REL CODE-ABS CODE-RET CODE-ABT 





The starting program is called ‘AUTO’. This is a menu-driven program that allows you to 
choose either targets (or expected value) or medians as the basis for evaluation. Upon 
selection, the appropriate program will be loaded. 


At this stage the data file is combined and the evaluation procedure is initiated. 
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On completion of the evaluation, a display of the evaluation diagram will appear on the 
screen. The diagram may be saved as a ‘pic’ file and the work sheet saved as a ‘wk1’ 


file. 


The study findings are presented in both tabular as well as graphical forms. For the 
former, relevant portions of the spreadsheet may be printed. For the latter, ‘pic’ files are 
imported into LOTUS FREELANCE®, scales suitably adjusted and acceptable 
performance zones marked. 


4.2 Data Format 


This program at present is limited to a maximum of forty participants. A similar program 
to include larger numbers (up to 100) is being developed. 


The first step is to ensure that the data is in a suitable LOTUS 123° format. As 
described earlier in section 2 the procedure evaluates a pair of analytical results from 
each participant. This pair of results and the laboratory identification (lab code) constitute 
the data set for each laboratory. The first data set is entered in Row 3 (Rows 1 & 2 are 
reserved for any titles that needs to be entered). The lab code is entered in column A, 
followed by the result of the lower concentration sample (in column B). The result of the 
higher concentration sample is entered in column C. The rest of the data sets are 
entered in a similar fashion on subsequent rows. An example of a suitable data format 
(File DATA1) is provided on the program diskette. 


Note: Since the program is limited to 40 participants, the data file cannot exceed row 43. 
Save the data file. 
4.3 Program Execution 


Retrieve the file "AUTO.wk1.” A menu will appear. Choose either target or median. 
The appropriate file will be retrieved and the cursor will be in cell 11. Combine your 
data file by pressing keys ALT & C simultaneously. Enter the data file name (include the 
path if data file is in a different directory) when requested and press RETURN key. On 
completion the data file is combined with the program file and the cursor will be in cell 11. 


NOTE: Ensure that program files are placed in the default directory. 


Press keys ALT & A simultaneously. Now relax! the program will take about 15 minutes 
to complete (depending on the number of participants, speed of your computer, variability 
of your data etc.,). On completion of the program you will see a graph on the screen. 
At this stage you may want to enter titles, make other changes and save the graph as 
a ‘PIC’ file. 
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Exit the graph menu to the spreadsheet and save the spreadsheet. Considerable disk 
space may be saved by extracting only the only relevant information (cells Ai to BL60) 
instead of saving the entire spreadsheet. 


On the finished spreadsheet many of the columns are hidden. Displayed columns from 
J to AY have all the information necessary for the tabular summary. Columns J & K 
contain the raw data. Columns L & M contain results of laboratories that were included 
in the final selection for the estimation of various ‘between laboratory statistics’ for both 
samples. The total error ( see section 2) for non acceptable participants is listed on 
column AW. Evaluation findings for each participant are listed in columns AX and AY. 


FIGURE 4.2 


MOE INTERLABORATORY STUDY 


LOW SAMPLE VS HIGH SAMPLE 


LOW SAMPLE (% OF MEDIAN) 





0 20 40 60 aa 100 120 140 160 160 200 


HIGH SAMPLE (% OF MEDIAN) 


Figure 4.2 is a typical ‘PIC’ graph that will appear on the screen. The graph is divided into 
four quadrants. Each participant is represented by a rectangular symbol. The diagonal 
line that goes through the origin represents the slope dependant bias. The other diagonal 
line represents the intercept dependant bias. The laboratories that have exceeded the 
warning limits are identified. The two triangular symbols on the Y-axis correspond to 2 
& 3 times repeatability (S,) from the centre respectively. These symbols serve as 
markers for drawing circles. 
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4.4 LOTUS FREELANCE? Refinement of ‘pic’ graphs. 


The ‘pic’ file is imported into LOTUS FREELANCE® to generate graphs with proper 
identification of the bias lines and acceptable performance zones. As mentioned earlier, 
a good working knowledge of LOTUS FREELANCE? is required to manipulate the various 


steps involved in producing a suitable graphical presentation. 


A brief outline of the major steps involved is given below. 


a) Import ‘pic’ graphs into LOTUS FREELANCE® 


b) Select all (SELECT;ALL). 


c) Scale down one axis of the graph so that the acceptable repeatability zone remains 


~— 


~— 


circular (EDIT; SIZE; NON-UNIFORM; etc). This will depend on whether evaluation was 
performed in absolute(concentration units) or relative (percentages of 
medias/targets) scale. This is a critical step and must be performed as accurately 
as possible. 


i) When performed in absolute scale, reduce the height (y-axis) of the graph 
until the angle between the intercept error line and the x-axis is equal to 45 
degrees. The unit lengths of both axis are identical in absolute 
(concentration) units. 


ii) When performed in relative scale, reduce the width (x-axis) of the graph 
until the angle between the slope error line and the x-axis is equal to 45 
degrees. The unit lengths of both axis are identical in relative scales. 


Draw circles with the point of intersection of the two median(or target) lines as the 
centres. The radus for the circle extend from the centre to the triangular symbols 
on the Y axis. 


Draw two bows, one on the lower left quadrant and the other on the upper right 
quadrant. The three points for each bow are the points of intersection of the outer 
circle with the two median ( or target) line and the 45 degree line. 


Select the outer circle (SELECT; NONE; SELECT; CYCLE; press arrow key on the 
number pad until outer circle is highlighted, press enter). Delete outer circle 
(REARRANGE; DELETE). The bows may appear fuzzy. The graph may be redrawn 
(VIEW; FULL) if you so desire. Select the two triangular symbols used to mark the 
radii and delete (REARRANGE; DELETE). 
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g) If more than one lab appears in the same area on the graph, the laboratory 
identification may not be legible. In such cases move the lab codes manually until 
the laboratory identification is clear. 


h) Label the two bias lines. 


i) Save file as a ‘drw’ file. 


4.5 Listing of LOTUS 123° programs 


The programs ‘EVALM’ , ‘CODE-ABS’ and ‘CODE-REL’ are listed line by line in sub- 
sections 4.5.1 to 4.5.3. Also included in the form of flow-diagrams an explanation of the 
fucnctions of various segments of each program. 


The programs ‘EVALT’, ‘CODE-ABT’ & ‘CODE-RET of the target mode are very similar 
to their counter parts of the median mode. Therefore these programs have not been 


duplicated. 
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4.5.1 EVALM 
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EVALM 


Cells A1-A35 
Calculates Mean, Median 


and Std.Dev. of both samples 
including all laboratories 


Cells A36-A54 
Screens high conc. sample for 
outliers and calculates Mean and 
Std.Dev.(SH) af selected labs. 


Cells AS6-a7i 
Screens low conc. sample for 
outliers and calculates Mean and 
Std. Dev.(SL) of selected labs 






Cell A72 
Calculates ratio of SH/SL 








Cells A73-A99 
Calculates the median of 


selected labs for both samples 







NO Cell A100 


IS SH/SL >2 








CODE-ABS 


CODE-REL 





CELL B109 


Combines the data file 
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Column D 


/dfh3..h43-1-1-45- Es er: 
{goto}146~Std.dev.~{right}~@std(j3..443)~ 
3 | {right }-@std (K3..K43)~{goto)i47- PS a ees 






/DSDh3 .M43~P443..443~D~G 


Donner mere ee must) 

nee ae 

nm 

air Ces =m are ee a aa 
EE RS RE ee 
nn maman ss 

Re em See 

re mm 









means 1 | - | 
Pe VE EP I NS 

CE fem | 
np | | 
en aen a 
a om À 
eme D 
enr ene 
WOM RS EE 
[20 farine je en ee 
LS RS EEE 
a a 
RS SNS RS 
D CSS SN RES 
RER EI ES 
LE 
ER ES RE ES 
D RS RE 
Se | 
ep | |... 
ET 
MMP corey ete ~eie(170=0,+175,4173)— RS 
D a 
ne es ee al 
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EVALM (cont. ) 


Column C | Column D 
{goto}m3~/rel3..m45~ 


| ROW | 
BRU 


















Column E 
{let diff, @abs (left-target) :value} 
{if diff<.100001*target}/cleft~~ 


/rncleft~{down}~ 


{down } 


À 4 | 


EN PE NE [| —*d 

E 

E 
EEE 


[let diff, @abs(left-target):value} 
[if diff<3.00001*val}/cleft-- 






/rncleft~{down}~ 


{down } | 


/cncnumber~m45~ 
{let number, 1+number} _ comp 


(if val=sd} {branch lo-sam} 


{let val, sd: value} 


49 | /rncleft~k3..k3~ eas acs 


50 | {goto}top~/rerange~ : /rerange~ 


{for e80,1, 41,1, iter} SE {branch cont} 





un 
LS] 


{down 2}-@std(range)- 





(down}~@avg (range) - 


(let val, sd/k48*3j48:value} 
(let val, sd/k48*3j48:value} 


57 /rvm45..m47-m45..m47- 


(if val=sd}{branch label} 
{let val,sd:value} 


second 








/rerange~ 





{branch low} 


58 /rncdiff~j70~/rnctop~13~ 


Œ 
Bes /rncsd-146..146- 

à 
amet | 





[gotoyaa=ok-ok~ 
Pants | _ | 

a 
CE [teeanch second) 
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EVALM (cont. ) 

































/rv145..147-m45..m47- 


71 |{goto}ll-low-sam-{right}hi-sam- 


E 
z 

7 
E 
a 
Sa 
mn a | 
eae CS 
ere 70-0,0175,4173)- À 
es 
a jan 
ne, es + 
Se a a 
Ea 
s0_[tcoroines-esmmsw- SCT OSSC—CS 
EEE 
A CE EE 
95 EE SE 
I EE RE 
a D 
EEE TOR CS SP 
RE EE EE RE 
EEE SSCS 
100 | {if j50>2}{branch C101} ee ae eee 
Re gotojatto= | | 
102 |/fececode-abs .wk1-  |/fececode-rer were | 
EEE os «un ppm ifbrench bMOME Me & = © 2 ee ee 





Page27 


4.5.2 CODE-ABS 
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CORES ASS 


Cells Bi-B8 
Ganlemlaves the Lota Error 


Cel1is-B9=-B21 
Calculates PD's to slope and 


intercept error lines 
select PD to intercept line 


repeat steps 1-20 to all labs 



















Gels, B22—BiaZ 
Determines the median of PD's to 


the intercept error line 





Cells G18-G24 
Calculates bias. selects appropriate PD's 





Calculates average PD and estimates 
average repeatability Sw 















Geils Boas —BS8 
Evaluates each pair of data 


Classifies errors (lines 37-58) & 
assvonssernrnor codes (lines 59-98) 





Cells B99-B113 
Calculates the appropriate variables, 





and plots diagram 


défine-plotting cCogrdinates, titles ete; 
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CODE-ABS 





/cn3..03-n3..043- 


{goto}p2-Tot.error-{right}theta-{right}slope” sine ra 


7 {right 2}slope.er.~{right}int’ pt.er.~ 


MAR 
DE 
RS) 
eon /r£f£f3-n3..043- 
[zal 
CN 


AE {goto}p3~@IF (J3=0,"-",@IF (K3=0,"-",@sqrt (((33- $1$48) 42) +((k3- ~$m$48)"2))))~_ 
See {goto }n60~@atan ($1$48/$m$48) *180/@pi~ 


DA Qif(j3<=$1$48#and#k3<=$m$48,1, @if (43>=$1548#and#k3>=$mS48, 2,0))~ 
{goto}r3~@if (t3>0, @abs (q3-$n$60) 1 q3+$n$60) ~ 
{right }~@if (t3>0, @abs (q3-45) , q3+45) ~ 


Here /rff1~q3..s3~ 










{goto}w3~@if (j3=0,"", @if (k3=0,"", @IF (P3=0,0, 
@IF (@COS (S3*@PI/180) *P3<4.5*$J$52, V3, @if (v3< 
US; va; u3)))))~ 










{GOTO} X3~@IF (j3=0,0, @if (k3=0,0, ,, Gif (W3>3*SIS5 
2,"",1)))~ 


{right }~@if (@isnumber (w3)=1,1,0)- 






21 Ve .X3-p3..x43- /Cw3..Y3-w3..Y43- 


/rvw3..w43-w3..w43- {GOTO}X46-@GSUM(X3..X43)-/C-Y46..Y46- 


Mr (goto) x60~ @sum(x3..x43)-/rvx60..x60-x60..x | {GOTO}J51~+Y¥46/X46*1.2533~ 


/dsdh3. .x43~pw3. .w43~d~g /rvj51..j52~j51..552~ _ 


{GOTO} w70~@MOD (x60, 2) ~ (BRANCH B142} 
{down}~@int ( (x60/2)+.5)~ 


221 {GOTO}w3~{DOWN w71-1} 
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CODE-ABS (cont. ) 


/c{down}~w73..w74~ 


{goto}w75~@avg (w73..w74) ~ 

{Goto} 452~@if (w70=0, +w75, +w73) ~ 

{BRANCH G126} : 

/dsdh3..x43~ph3..h43~a~g | 
{branch b146} 


{goto}z3~+@IF (@ISNUMBER (P3)=0,"", @int (p3/$45$51*10) /10) ~{right}+u3/$j$51~{right }+v3/$j5$51 
~/rff1~z3. .ab3~ 


{right} @IF (@ABS (@ABS (33) -SL$48) /$3$51<2,1, @IF (@ABS (@ABS (k3) -$M$48) /$3$51<2,1,0))~ 


(right }@if (t3=1#OR#t3=2, 0, @if (p3=0, 0, @if (z3<=2,1, @if (z3>2#and#z3<=3, 2, @if (ac3=1,15,16))) 
))~ 


{right }@if (t3=2#or#t3=0, 0, @IF (z3<=4.5, 0, @if (q3<45,0, @IF (Ab3<1, 4, @if (ac3=1,15, @IF (Ab3>1#A 
ND#Ab3<2,3,8))))))~ 


{right } @if (T3=2#or#T3=0, 0, @IF (23<=4.5,0, @if (q3>SNS$60, 0, @IF (AA3<1, 6, @if (aC3=1, 15, GIF (AA3> 
1#AND#AA3<2,5,8))))))~ 


{right } @if (T3=2#or#T3=0, 0, @IF (Z3<=4.5,0, @if (q3<=SNS$ 60#OR#Q3>=45, 0, @IF (AB3>AA3, 0, @IF (AB3< 
1,4, @IF (AB3>1#AND#AB3<2,3,7))))))~ 


{right} @if (T3=2#or#T3=0, 0, @IF (23<=4.5,0, @if (q3<=SNS$60#OR#03>=45, 0, @IF (AA3>AB3, 0, @IF (AA3< 
1,6, GIF (AA3>1#AND#AA3<2,5,7))))))~ 


(right }@if (t3=1#or#t3=0, 0, @IF (z3<=4.5,0, @if (q3<45, 0, @IF (Ab3<1, 10, @if (ac3=1,15, GIF (Ab3>1# 
AND#Ab3<2,9,14))))))~ 


{right }@if (T3=1#or#T3=0, 0, @IF (Z23<=4.5,0, @if (q3>$N$60, 0, @IF (AA3<1, 12, @if (aC3=1,15, @IF (AA3 
>1#AND#AA3<2,11,14))))))~ 


{right } @if (T3=1#or#T3=0,0,@IF (Z3<=4.5,0, @if (q3<=SNS 60#OR#Q3>=45, 0, GIF (AB3>AA3, 0, GIF (AB3< 
1,10, @IF (AB3>1#AND#AB3<2,9,13))))))~ 


{right} @if (T3=l#or#T3=0, 0, @IF (23<=4.5, 0, @if (q3<=$N$60#OR#03>=45, 0, @IF (AA3>AB3, 0, @IF (AA3< 
1,12, @IF (AA3>1#AND#AA3<2,11,13))))))~ 


{right } @if (t3=2#or#t3=0, 0, @IF (z3<=3#OR#2Z3>4.5,0, @if (q3<45,0, GIF (Ab3<1,18, @if (ac3=1,29,@I 
F (Ab3>1#AND#Ab3<2,17,22))))))~ 


{right} @if (T3=2#or#T3=0, 0, @IF (23<=3#OR#Z3>4.5,0, @if (q3>SNS60, 0, @IF (AA3<1, 20, @if (aC3=1, 29 
, @IF (AA3>1#AND#AA3<2,19,22))))))~ 


{right } @if (T3=2#or#T3=0, 0, @IF (Z23<=3#OR#23>4.5,0,@if (q3<=SNS 60#OR#Q3>=45, 0, GIF (AB3>AA3,0, 
@IF (AB3<1,18, @IF (AB3>1#AND#AB3<2,17,21))))))~ 



























































{right } @if (T3=2#0r#T3=0, 0, @IF (23<=3#OR#Z3>4.5,0, @if (q3<=SNS60#OR#Q3>=45, 0, @IF (AA3>AB3,0, 
@IF (AA3<1, 20, @IF (AA3>1#AND#AA3<2,19,21))))))~ 
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CODE-ABS (cont. ) 








52 | {right }@if (t3=l#or#t3=0, 0, @IF (z3<=3#OR#Z23>4.5, 0, @if (q3<45, 0, @IF (Ab3<1, 24, @if (ac3=1, 29, @I 
F (Ab3>1#AND#Ab3<2, 23, 28))))))~ 


53 {right }@if (T3=l#or#T3=0, 0, @IF (Z23<=3#OR#23>4.5,0,@if (q3>$NS$60, 0, @IF (AA3<1, 26, @if (aC3=1, 29 


54 {right}@if(T3=1#or#T3=0, 0, @IF (Z3<=3#OR#23>4.5,0, @if (q3<=$N$60#OR#Q3>=45, 0, @IF (AB3>AA3, 0, 
@IF (AB3<1, 24, @IF (AB3>1#AND#AB3<2,23,27))))))~ 





55 {right } @if (T3=l#or#T3=0, 0, @IF (Z3<=3#OR#Z3>4 .5,0, @if (q3<=$N$60#OR#03>=45, 0, @IF (AA3>AB3, 0, 
@IF (AA3<1, 26, @IF (AA3>1#AND#AA3<2,25,7))))))~ 


56 {RIGHT } @IF (T3=0, 0, @IF (23<=3,1,0))~ 
Si {right }~@IF (@ISNUMBER (P3)=0,0,@sum(aD3..aU3))- 


58 | (RIGHT) @IF (AV3>1, 23, "")~/RFF1~AW3. .AW3~ 
59 /cz3..aW3-z3..aW43-/rnccode-b173..b174- 


/rvav3..aw43-av3..aw43- 

61 /RNCLEFT-AV3..AV3-{goto}ax3- 

62 {for a200,1,41,1,B174} 

63 {branch b207} 

65 {IF LEFT=1}A~ {RIGHT} ACCEPTABLE PERFORMANCE~ { LEFT } 

(IF LEFT=2}AI~{RIGHT}WARNING SLIGHT IMPRECISION~{LEFT} : 

67 {IF LEFT=3}Li~{RIGHT}BIASED LOW, POSSIBLE INTERCEPT PROBLEM~ {LEFT} 
{IF LEFT=4}LI~{RIGHT}BIASED LOW, PROBABLE INTERCEPT PROBLEM~ {LEFT} 
{IF LEFT=5}Ls~{RIGHT} BIASED LOW, POSSIBLE SLOPE PROBLEM~{LEFT} 

70 {IF LEF T=6}LS~ {RIGHT} BIASED LOW, PROBABLE SLOPE PROBLEM~{LEFT} 


TE {IF LEFT=7}L~{RIGHT}BIASED LOW-{LEFT} : 
2. = 


{IF LEFT=8}Le~{RIGHT}BIASED LOW OR ERRATIC~{LEFT} 


73 {IF LEFT=9}Hi~{RIGHT}BIASED HIGH, POSSIBLE INTERCEPT PROBLEM~{LEFT} 


7 


74 {IF LEFT=10}HI-{RIGHT}BIASED HIGH, PROBABLE INTERCEPT PROBLEM-{LEFT} 
75 {IF LEFT=11}Hs~{RIGHT}BIASED HIGH, POSSIBLE SLOPE PROBLEM-{LEFT} 

76 {IF LEFT=12}HS~{RIGHT}BIASED HIGH, PROBABLE SLOPE PROBLEM~{LEFT} 

a {IF LEFT=13}H~{RIGHT}BIASED HIGH~{LEFT} 


78 {IF LEFT=14}He~{RIGHT}BIASED HIGH OR ERRATIC~{LEFT} L 
79 (LE LEFT=15}OC-{RIGHT}ONE RESULT ERRATIC-({LEFT} , 
{IF LEFT=16}ER~{RIGHT}BOTH RESULTS ERRATIC~{LEFT} 


81 {IF LEFT=17}WLi~{RIGHT}WARNING:BIASED LOW, POSSIBLE INTERCEPT PROBLEM~ {LEFT} 
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CODE-ABS (cont. ) 


mmm 
82 
8: 
24 
as 


86 {IF LEFT=22}WLe~{RIGHT}WARNING:BIASED LOW OR ERRATIC~{LEFT} 






















87 {IF LEFT=23}WHi~{RIGHT}WARNING:BIASED HIGH, POSSIBLE INTERCEPT PROBLEM~ {LEFT} 
88 {IF LEFT=24}WHI~{RIGHT}WARNING:BIASED HIGH, PROBABLE INTERCEPT PROBLEM~ {LEFT} 
89 {IF LEFT=25}WHs~{RIGHT}WARNING:BIASED HIGH, POSSIBLE SLOPE PROBLEM~ {LEFT} 

90 {IF LEFT=26}WHS~{RIGHT}WARNING:BIASED HIGH, PROBABLE SLOPE PROBLEM~ {LEFT} 

91 {IF LEFT=27}WH~{RIGHT}WARNING:BIASED HIGH~{LEFT} 

92 {IF LEFT=28}WHe~{RIGHT}WARNING:BIASED HIGH OR ERRATIC~{LEFT} 


93 
24 
95 
96 
98 
99 
00 
01 {goto}bg44-+m48-{down}+m48-{right}0-{up}2*148- 

02 
03 
04 
os 
06 


07 {goto}bg51~+m48+148~{right 4}~+148*2~ 
08 {goto}bg52~+m48~{right 5}+148+(2*j51) ~ 


09 {goto}bg53~+m48~{right 5}+148+(3*j51)~ 


/grgtxxbg3. .bg53~abf3..bf53~bbh3. .bh53~cbi3. .bi53~dbj3. .bj53~ebk3. .bk53~fb13. .b153~ 





ital ocfglasfsqsymuL48*2~qsxmuM48*2~q 


dabe3. .be53~aq 


tfINTERLABORATORY STUDY: K-S PROCEDURE ~TS LOW CONC.SAMPLE VS HIGH CONC. SAMPLE ~TXHIGH 
CONC.SAMPLE ~TY LOW CONC. SAMPLE ~QV 
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4.5.3 CODE-REL 
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CODE— REL 


Cells ETES 
Transform results into percentages of median 


Galeulatels the Total Error 







Cells B9-B22 
Calculates PD's to slope and 
Intercept errorsliines 
select PD to slope line 
repeat steos 1-21 to all labs 










Cells B22-B33 
Determines the median of PD's to 












the intercept error line 





Cells G18-G24 
Calculates bias, selects appropriate PD's 


Calculates average PD and estimates 
average repeatability Sw 








GellseBa4—-B100™ 
Evaluates each pair of data 





CcHassifiesterrorsm(liines 39-60 )..& 






assagnswerror?) codes (lines: 61-100) 








Gells "E101-E17175 
Calculates the appropriate variables, 






Gefiane=plotting coord snaves>s etitilessette 
and plots diagram 
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CODE-REL 










INE 
(goto}n3~ 
@IF ($4$50>2, +J3*100/$L$48, +53) ~ {Right }~ ir 





@IF ($3$50>2, pis TOUS eee +k3) ~ 


eel /cn3..03-n3..043- ; : =o ee 
aes /rff0~n3. .043~ rs a 
ENS {goto}p2~Tot.error~{right}theta~{right}slope*~{right}int’ pt*~ =o a 


{right 2}slope.er.~{right}int’ pt.er.~ | | 1 oe 
{goto}p3~@IF (J3=0, "—", @IF (K3=0, "-", @sqrt ( ( (N3-100) *2) + ( (03-100) *2))))~ 


ae {GOTO }N59~+ ( (M48-L48) /M48) ~ ; DE 
EC {goto}n60-@atan((1-n59))*180/@pi- 
{goto}q3~@asin (@INT (@abs (n3- 100) /p3*10000) /10000) *180/@pi~ 









{goto}t3~ 










{goto}r3~@if (t3>0, @abs (45-q3) ,q3+45) ~ 


(right }~@if (t3>0, @abs (90- (q3+$n$60) ) , 90-@abs (q3-$n$60) ) ~ oe = 4 







{GOTO}W3~@IF (33=0,"",@IF(K 
3=0,"", @IF (P3=0,0, @IF (@COS 
(R3*@PI/180) *p3<4.5*$J$52, 
u3, @IF (u3<v3,u3,v3)))))~ 









{GOTO}X3~@IF (33=0, 0, @if (k3 
=0,0, @if (W3>3*S$J$52,"",1)) 
)~ 


ee {right }~@if (@isnumber (w3)=1,1,0)- {GOTO} Y3~@IF (W3>3*$J$52,"" 
fai Tee 


ib aa! est wat eset SONT) | fows Aysewsievas~ | .Y3-w3..Y43- 


/cp3..x3-p3..x43- {GOTO}X46-ESUM(X3..X43)-/C 
-Y46..Y46- 
| 23 | sewwa. .w43-w3..w43- {GOTO} J51~+¥46/X46*1.2533~ 


{goto}x60-@sum(x3..x43)- {BRANCH B143} 

























/dsdh3. 
{GOTO }w70~@MOD (x60, 2) ~ 





.-X43-pw3..w43-d-g 







27 
28 





{down}-@int ((x60/2)+.5)- 






{GOTO}w3~{DOWN w71-1} 





CODE-REL (cont. ) 


a — 
| spirale, Ea 


dl 


{goto}z3-+@int (p3/$4$51*10) /10~ {right }+u3/$j$51~{right }+v3/$4j$51~/rffl1~ 
z3..ab3~ 


{right } @IF (@ABS (n3-100) /$4$51<2,1, @IF (@ABS (03-100) /$3$51<2,1,0))~ 


{right} @if (t3=1L#OR#t3=2, 0, @if (p3=0,0, @if (z3<=2,1, @if (z3>2#and#z3<=3,2,@ 


2 
30 
sil 
32 
33 
34 
35 
36 
37 





_ 
H 





if (ac3=1,15,16)))))~ 
{right } @if (t3=2#or#t3=0, 0, @IF (z3<=4.5, 0, @if (q3< (90-$NS$60) , 0, @IF (Ab3<1, 4 


n= 
i) 









, @if (ac3=1,15, @IF (Ab3>1#AND#Ab3<2,3,8))))))- 


{right } @if (T3=2#or#T3=0, 0, @IF (23<=4.5,0, @if (q3>45,0, @IF (AA3<1, 6, @if (aC3 
=1,15, @IF (AA3>1#AND#AA3<2,5,8))))))~ 


{right } @if (T3=2#or#T3=0, 0, @IF (23<=4.5,0, @if (q3>=(90-$N$60) #OR#Q3<=45,0, 
@IF (AB3>AA3, 0, @IF (AB3<1, 4, @IF (AB3>1#AND#AB3<2,3,7))))))~ 


{right} @if (T3=2#or#T3=0, 0, @IF (23<=4.5,0, @if (q3>=(90-$N$60) #OR#Q3<=45,0, 
@IF (AA3>AB3, 0, @IF (AA3<1, 6, @IF (AA3>1#AND#AA3<2,5,7))))))~ 


{right} @if (t3=lL#or#t3=0, 0, @IF (z3<=4.5,0, @if (q3< (90-$NS$60) , 0, @IF (Ab3<1,1 
0, @if (ac3=1,15, @IF (Ab3>1#AND#Ab3<2,9,14))))))~ 


{right} @if (T3=l#or#T3=0, 0, @IF (Z3<=4.5,0, @if (q3>45,0, @IF (AA3<1,12, @if (ac 
3=1,15, @IF (AA3>1#AND#AA3<2,11,14))))))~ 


{right } @if (T3=lL#or#T3=0, 0, @IF (Z3<=4.5,0, @if (q3>=(90-SNS60) #OR#Q3<=45, 0, 
@IF (AB3>AA3, 0, @IF (AB3<1,10, @IF (AB3>1#AND#AB3<2,9,13))))))~ 


> 
un 






ea 








{right} @if (T3=1#or#T3=0,0,@IF(Z3<=4.5,0,@1f (q3>=(90-SNS60) #OR#Q3<=45, 0, 
@IF (AA3>AB3, 0, @IF (AA3<1,12, @IF (AA3>1#AND#AA3<2,11,13))))))~ 


{right} @if (t3=2#or#t3=0, 0, @IF (z3<=3#OR#Z3>4.5,0, @if (q3< (90-SN$60),0, @IF 
(Ab3<1,18, @if (ac3=1, 29, @IF (Ab3>1#AND#Ab3<2,17,22))))))~ 


{right } @if (T3=2#or#T3=0, 0, @IF (Z3<=3#OR#Z3>4.5,0, @if (q3>45,0, @IF (AA3<1,2 
0, @if (aC3=1, 29, @IF (AA3>1#AND#AA3<2,19,22))))))~ 


{right} @if (T3=2#or#T3=0, 0, @IF (Z3<=3#OR#23>4.5,0,@if (q3>= (90-SNS60) #OR#Q 
3<=45,0, @IF (AB3>AA3, 0, @IF (AB3<1,18, @IF (AB3>1#AND#AB3<2,17,21))))))~ 


ou 
Oo 






D 






un 
to 


{right } @if (T3=2#or#T3=0, 0, @IF (Z23<=3#OR#Z3>4.5,0, @if (q3>=(90-SNS60) #OR#Q 
3<=45,0, @IF (AA3>AB3, 0, @IF (AA3<1, 20, @IF (AA3>1#AND#AA3<2,19,21))))))~ 


on un c= c= Ww 
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{right} @if (t3=l#or#t3=0, 0, @IF (z3<=3#OR#Z3>4.5,0, @if (q3< (90-SNS$60) ,0, GIF 
(Ab3<1, 24, @if (ac3=1, 29, @IF (Ab3>1#AND#Ab3<2, 23,28) )))))~ 


{right} @if (T3=1#or#T3=0, 0, @IF (Z3<=3#OR#Z3>4.5,0, @if (q3>45,0, @IF (AA3<1,2 
6, @if (aC3=1, 29, @IF (AA3>1#AND#AA3<2, 25,-28))))))~ 


{right }@if (T3=1#or#T3=0, 0,@IF (Z3<=3#OR#23>4.5,0,@if (q3>=(90-$N$ 60) #OR#Q 
3<=45,0,@IF (AB3>AA3, 0, @IF (AB3<1,24,@IF (AB3>1#AND#AB3<2,23,27))))))~ 


{right } @if (T3=1#or#T3=0, 0, GIF (Z3<=3#OR#Z3>4.5,0, @if (q3>=(90-SNS60) #OR#Q 
3<=45,0, @IF (AA3>AB3, 0, @IF (AA3<1, 26, GIF (AA3>1#AND#AA3<2,25,7))))))~ 


| sa" | {RIGHT} @IF (T3=0, 0, @IF (Z23<=3,1, 0) )~ 
{right }~@IF (@ISNUMBER (P3) =0,0, @sum(aD3..aU3))- 


eo {RIGHT} @IF (AV3>1, Z3, "") ~/RFF1~AW3. .AW3~ 
DT /cz3..aW3-z3..aW43-/rnccode-b171..b171- 


BEN /rvav3..aw43-av3..aw43- 
Les [ymnensrr-avs. .avs-igtoagse 0 à | 
| ER RP 
{IF LEFT=1}A~{RIGHT}ACCEPTABLE PERFORMANCE~{LEFT) 

{IF LEFT=3}Li~{RIGHT}BIASED LOW, POSSIBLE INTERCEPT PROBLEM~{LEFT} 
{IF LEFT=4}LI~{RIGHT}BIASED LOW, PROBABLE INTERCEPT PROBLEM~{ LEFT} 
{IF LEFT=5}Ls~{RIGHT}BIASED LOW, POSSIBLE SLOPE PROBLEM~ {LEFT} 


| {IF LEFT=6}LS~{RIGHT}BIASED LOW, PROBABLE SLOPE PROBLEM~ {LEFT} 


{IF LEFT=7}L~{RIGHT}BIASED LOW~{LEFT} 

{IF LEFT=8}Le~{RIGHT}BIASED LOW OR ERRATIC~{LEFT} 
(IF LEFT=9}Hi~{RIGHT}BIASED HIGH, POSSIBLE INTERCEPT PROBLEM~{ LEFT} 
{IF LEFT=10}HI~{RIGHT}BIASED HIGH, PROBABLE INTERCEPT PROBLEM~ {LEFT} 
{IF LEFT=12}HS~{RIGHT}BIASED HIGH, PROBABLE SLOPE PROBLEM~ {LEFT} 

{IF LEFT=13}H~{RIGHT}BIASED HIGH~ {LEFT} 

| feo "| {IF LEFT=14}He~{RIGHT}BIASED HIGH OR ERRATIC~{LEFT} 


{IF LEFT=15}0C~{RIGHT}ONE RESULT ERRATIC~{LEFT} | 
DE {IF LEFT=16}ER~{RIGHT}BOTH RESULTS ERRATIC-{LEFT} 


83 {IF LEFT=17}WLi~{RIGHT}WARNING:BIASED LOW, POSSIBLE INTERCEPT 
ne {LEFT} 





CODE-REL (cont. ) 


{IF LEFT=18}WLI~{RIGHT}WARNING:BIASED LOW, PROBABLE INTERCEPT 
PROBLEM~ { LEFT} 


85 {IF LEFT=19}WLs~{RIGHT}WARNING:BIASED LOW, POSSIBLE SLOPE 
PROBLEM~ {LEFT} 


{IF LEFT=20 } WLS~ {RIGHT} WARNING: BIASED LOW, PROBABLE SLOPE 
PROBLEM~ { LEFT} 


87 {IF LEFT=21}WL~{RIGHT}WARNING:BIASED LOW~{LEFT} 


88 {IF LEFT=22}WLe~{RIGHT}WARNING:BIASED LOW OR ERRATIC~ {LEFT} _ 


{IF LEFT=23}WHi~{RIGHT}WARNING:BIASED HIGH, POSSIBLE INTERCEPT 
PROBLEM~ { LEFT} 


{IF LEFT=24}WHI~{RIGHT}WARNING:BIASED HIGH, PROBABLE INTERCEPT 
PROBLEM~ { LEFT} 


91 {IF LEFT=25}WHs~{RIGHT}WARNING: BIASED HIGH, POSSIBLE SLOPE 
PROBLEM~ { LEFT} 


92 {IF LEFT=26}WHS~{RIGHT}WARNING:BIASED HIGH, PROBABLE SLOPE 
PROBLEM~ { LEFT} 
93 {IF LEFT=27}WH~{RIGHT}WARNING:BIASED HIGH~{LEFT} 
{IF LEFT=28}WHe~{RIGHT}WARNING:BIASED HIGH OR ERRATIC~{LEFT} 


95 {IF LEFT=29}WOC~{RIGHT}WARNING:ONE RESULT ERRATIC~{LEFT} 












DER 









{IF LEFT=30}WER-{RIGHT}WARNING:BOTH RESULTS ERRATIC-{LEFT} 


/RNCLEFT-{DOWN}- 









110 
111 


112 /grgtxxbg3..bg53~abf3. .bf53~bbh3. .bh53~cbi3. .bi53~dbj3. .bj53~ebk3. .bk53 
web). DLS ~ 


{goto}bg46-0-{right 2}100~ 
{goto}bg47~200~{right 2}100~ 


97 
100 
101 
102 
103 
104 
105 
106 
107 
108 
109 
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CODE-REL (cont. ) 


Ce a Scere lana. seen EE 
ocfglasfsqsymu200~L0~qsxmu200~L0~q 


aR 


TS tfMOE INTERLABORATORY STUDY ~TS LOW SAMPLE VS HIGH SAMPLE -TXHIGH 
SAMPLE (% OF MEDIAN)~TYLOW SAMPLE (% OF MEDIAN)-QV 


















